Problem Statement

Samsung.com (US) conducts many experiments annually, ensuring the conversion rate consistently grows. Every growth strategy was tested with experimentation, and we built an e-commerce site that grew 40x between 2019 and 2022. Many features include Horizontal vs. vertical Configurator, Express checkouts, Top and Bottom Continue buttons, and FBT (Frequently Bought Together). These are simple examples that were tested using experimentation. Some user experiences were successful with Design 'A' and some with Design 'B,' impacting the two main metrics: Conversion Rate (Orders/Checkouts) and Lower Cart abandonment (Orders/Carts). In this project, I will provide experimentation details about landing users to checkout out vs add to cart after "Buy Now" is configured.

This experiment does not list out data as do not have access to data

Overview

Currently, Samsung.com/us faces a high bounce rate of 38%, and many are leaving the funnel after adding the signature phones to the cart. Thus, the conversion rate dropped to 21%, which is historically a low point, with a cart abandonment rate close to 88%, meaning users abandon 88% of the carts created. I am planning on conducting an experiment to direct users to land on the Checkout page after the Buy Now configuration vs. directing them to the Add To Cart page and then to checkout. My stakeholders are the sales and marketing team, the Operations team, and the e-commerce category managers.

Hypothesis

Experiment: Comparing conversion rates for two user journeys:

• Control: Users land directly on the checkout page.
• Test: Users are directed to the "add to cart" page and then to checkout.

Null Hypothesis (H0): There is no significant difference in conversion rate between users who land directly on the checkout page (control) and users who are directed to the "add to cart" page before checkout (test).

Explanation:

• The null hypothesis assumes that the change in the user journey (adding the "add to cart" step) does not impact the conversion rate.
• This means that any observed difference in conversion rates between the control and test groups is due to random chance, not the change we made.

In simpler terms, Directing users to the "add to cart" page before checkout will not significantly increase or decrease, not the number of users completing a purchase.

Purpose of the Experiment:

The experiment aims to disprove the null hypothesis. If the test group shows a statistically significant improvement in conversion rate compared to the control group, we can reject the null hypothesis and conclude that the change in user journey positively impacts conversions.

Goal

Our site has 6 million weekly visitors; upper-funnel to middle-funnel conversion is only 4%. With Growth Targets set to a Conversion Rate of 24 to 25%. Upon checking the Adobe analytics data, most users left after adding their mobile phones to the cart. Our main goal is to improve the conversion rate of checkouts and lower cart abandonment after adding the

Success Metric

Conversion Rate = [Total Orders/Total Checkouts] * 100

​

Lower Cart Abandonment Rate (LCA Rate) = 1 - (Orders/Total Carts Created) *100

In both Control and Test measures, Checkouts, Carts, Orders(no of Thank you pages), Conversion Rate, and LCA Rate

​

Experimentation Design

What are you testing

Variation Design

Audience & Sampling Sizing

​

We conducted the test with a 50-50 split between Control and Test. We implemented a randomization that removes bias and ensures that the baseline condition for both groups is the same before the AB Test. As we applied Randomization, the split won't always be exactly 50-50 split given the randomization.

Implementation & A/A Testing

We experimented on desktop web and mobile web. Total users are split with a randomization algorithm with a specific campaign ID under which all the Weblog data was logged with a flag of 0 (control) and 1(Test)

​

​

​

Control Flow:

1-->2-->4

Test flow :

1-->2-->3-->4

Sample size Calculation:

​

​

Post Experimentation

Experiment Result

We found that users significantly created orders during the test when the workflow was 1-->2-->4. However, our cart abandonment rate was not improving as some saved bookmarks pointed them to Add to Cart. A small new experimentation within this test was conducted to reduce cart abandonment by adding a "Paypal" button under checkout on the add to cart page. see the below:

​

Release Decision

We didn't follow the statistical significance route as our stakeholders were seeing a more than 5-point increase in the Conversion Rate, and this whole experiment was run for a complete four weeks. Finally, stakeholders are leaning toward both changes

1. Follow the flow 1-->2-->4
2. In the case of add-to-cart, use the Paypal button.

Learnings

1. You do not need to be too orthodox in conducting experiments when the outcome is clear and showing the bottom line impact to success metrics
2. Combining two or more features in the same test will not provide good results. Our Paypal adoption in the add to cart area has significantly delayed our implementation time time to market has been impacted.

Next Steps

We wanted to reduce the checkout inputs and provide hyper-personalization.

Stakeholder Management

Stakeholders were stuck at LCA Rate improvement while ignoring the Conversion Rate improvements. I have presented same things again and again with simple calculations and YOY improvements were the biggest key in selling the final design.

​

​